Conversation
This commit improves I/O related to artefact checksum computation. It easily brings great performance improvements. I ran `sbtix-gen-all` on typelevel cats: - Before it took 1136s - Now it takes 113s This is 10x faster on this particular project.
|
Wow, I have no idea why it was taking the round-trip through that file. That should absolutely be dropped. The semaphore is there to avoid annoying the Maven repo with too many simultaneous requests. It can probably be increased a bit safely, but I wouldn't be comfortable removing it entirely. I'm not comfortable depending on sun.* internals (though a 10x improvement would definitely be worth having a fast path for). I'm also curious about what difference it would make to steam bytes into the hash rather than do it all at once in the end. |
|
Also: I'm in the process of moving the project to GitLab due to the recent acquisition. Issues and PRs are supposed to be migrated, but I'm not sure how it handles changes after the migration has started. |
|
@vquintin awesome! I'll try to have a look at this as well in the afternoon :-)
👍 👍 👍 |
|
Is the semaphore there because Regarding the sun.* import. I did this to get something quickly. I can remove it and instead:
I think streaming would bring two things:
Don't forget to communicate when the GitLab migration is done so we can move there 👍 |
|
The migration is now done: https://gitlab.com/teozkr/Sbtix/merge_requests/40 |
This PR aims to make sbtix generation faster.
It depends on #37.
The interesting bit is the last commit (2d7125c).
The modifications are short, I only made the I/O more efficient.
I ran
sbtix-gen-allon typelevel cats:This is 10x faster on this particular project.